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ABSTRACT 

Aural comprehension tests, which are designed so 
that the student can give non-vocal reactions to orally-presented 
problems are valuable for two reasons: they provide the teacher of 
English with an objective and accurate measure of the pupils' 
comprehension and can also be given simultaneously to a large group 
of students. According to the author, it is very important that this 
kind of test be properly prepared, taking into consideration that the 
structured oral approach must use meaningful alternatives based on 
expressions already practiced in class. These tests are best 
administered with a tape recorder or a language laboratory to 
eliminate possible subjectivity in the teacher's voice, and are 
apparently most successful when the answer form is completion, single 
response, or supplying a sentence to describe a situation. When using 
contrastive elements, it is best not to give widely varying 
alternatives; although three alternatives are preferable some 
structures will not permit this. The author further suggests ways of 
preparing the test script so it will be meaningful and ways of 
checking the reliability of the test itself. He emphasizes that the 
aural comprehension test cannot measure a student's speaking ability. 
(PB) 
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Aural comprehension tests can be defined as language 
tests which require of the student non-vocal reactions to 
problems presented orally. When properly constructed, such 
tests are &n excellent means of giving the teacher of spoken 
English an objective and accurate quantitative assessment of 
his students’ progress in aural ability; and, when presented 
in multiple choice form, they are easy to give simultaneous- 
ly to a large number of students with much time saved in the 
task of grading. 

An example familiar to most English teachers in Japan 
is the Sound Perception Test used to test a student's abili- 
ty to distinguish problem phonemes in the English sound sys- 
tem. In its most common form, the student hears three sen- 
tences, one of which differs from the other two only in the 
phoneme contrast being tested. The student might hear, for 
example: 




A. Did you write it? 

*B. Did you light it? 

C. Did you write it? 

If he is able to distinguish the phonemes /r/ and /l/, 
he will then mark the letter B on his answer sheet. Similar 
techniques can also be used to test discrimination of other 
phonemic contrasts such as sentence stress and intonation. 

It is not the purpose of this paper to deal with aural 
comprehension tests on sound structure. The focus is rather 
on the testing of grammatical structure, and of vocabulary 
irisofar as problem words ‘are related to specific structures. 
The purpose, then, is to discuss the various factors which 
must be taken into account when preparing aural comprehen- 
sion tests on grammatical structure and to provide guide- 
lines for teachers who wish to prepare their own tests of 
this type. Since the field of testing is very wide, the dis- 
cussion will be limited to the case of the teacher using a 
structured oral approach with one or more groups of students 
at beginning or intermediate levels. This is probably the 
most common situation in Japanese schools which have adopted 
a linguistic approach to English teaching, and it is also 
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the situation in which scientific testing methods can be 
most beneficial. 

At this stage, it is important to point out that the 
type of test to be considered here is one in which meaning 
plays an important role. In this aspect it differs consider- 
ably from the sound perception test briefly described above. 
In such a test, many linguists could agree that it would be 
equally valid to test auditory discrimination between /r/ 
and / 1/ by using, for example: 

I want a runk. - I want a lunk. - I want a runk. 

the fact that ”runks" and "lunks" do not correspond to any 
reality being immaterial. In the case of grammatical struc- 
ture tests, however, it is not usually possible, nor advis- 
able, to try to isolate structure in a chemically pure state 
divorced from meaning. In an example such as: 

He took up English... A. well B. hard C. gladly 

it is obvious that, in order to select the correct comple- 
tion C, the student must have a very clear understanding of 
the meaning of the verb take up and of the usage of the ad- 
verbs given. On the other hand, meaning must be tested in 
relation to specific patterns, and that is why it is neces- 
sary to use a structural analysis as the basis for tests of 
this type. 

WHAT TO TEST 

From the foregoing it is clear that the material to be 
tested must be described in terms of grammatical structures 
or patterns, NOT on the basis of the number of vocabulary 
items taught or language situations encountered. In the case 
under consideration, the teacher will know that the struc- 
tures to be tested are those which to date have already been 
practised in the classroom. It would be linguistically un- 
sound and certainly unfair to the students to set problems 
involving structures not yet formally dealt with, and then 
trust to luck that the students had somehow become familiar 
with them outside of the class. 

Since it is assumed that the method of teaching is lin- 
guistically structured, the deciding of what to test should 
not be difficult, as the textbook would presumably present 
the different patterns systematically. 
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MODE OF PRESENTATION 

In order to clarify the different techniques of testing 
to be described, it is helpful to point out at this stage 
that the type of test envisaged is a recorded one to be 
given in a language laboratory or with the aid of a class- 
room tape recorder of sufficiently high fidelity. The advan- 
tage of this is that it removes another subjective element 
from the test, for, if it is read out in the classroom, there 
is a danger that sympathy for the students may more or less 
unconsciously cause the the teacher to slow down his deliv- 
ery or artificially emphasize certain features. The machine 
on the other hand is inexorable and admits of no appeal. An- 
other advantage is that the same test can be given to sever- 
al classes under standard conditions and be used several 
times in succeeding courses. In schools with several teach- 
ers giving the same course it would perhaps be better to 
have someone who is not actually teaching record the test so 
that the conditions are equal for all students. If this can- 
not be done, it is usually possible to arrive at some com- 
promise. 



FORM OF PRESENTATION 

Assuming that the test is based on the multiple choice 
system with three alternatives A, B and C, the questions 
eg a be presented in several different forms, the choice of 
form being usually determined by the particular structure 
being tested. A number of different forms will now be con- 
sidered. 

A. Choose the correct completion. 

The students hear the first part of a sentence which is 
then completed in three different ways, only one of which is 
correct. The following examples illustrate this technique: 

(1) I watched television.... 

A. tomorrow *B. last night C. now 

(2) We've studied English.... 

*A. for one year B. one year ago 
C. after one year 

(3) Mr. Tanaka took up English.... 

A. well B. harti *C. gladly 
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(4) X don't have to get up early.... 

*A. but I intend to B. but X must 
C. but X don't want to 

(5) I must go now, but I wish.... 

A. I mustn't *B. I didn't have to 
C . I wouldn ' t 

(6) I'd go to Hawaii.... 

A. if I have enough money 

B, if I'd had enough money 
*C. if I had enough money 

(7) I'm very keen.... 

*A. on going B. for going C. at going 

(8) You heard the news.... 

A. don't you? *B. didn't you? C. aren't you? 

(9) There won't be any classes next week.... 

A. won't there? B. will they? *C. will there? 

(10) She never arrives on time.... 

*A. does she? B. doesn't she? G. isn't she? 

(11) The students repeat the sentences.... 

A. doesn't he? *B. don't they? C. do they? 

(12) The teachers were busy when we came..,, 

A. didn't we? B. don't they? *C. weren't they? 

These examples illustrate the testing of the students 
recognition and understanding of tenses (1, 2), adverbs of 
manner (3), connected sentences (4, 5), verb forms employed 
after VISE and in conditional sentences (5, 6), and preposi- 
tions (7) . The use of tag questions (8-12) is especially 
useful for testing verb forms, the distinction between sin- 
gular and plural (11), and the recognition of the main verbs 
in complex sentences (12) . 

The completion technique can be modified by having the 
students choose one incorrect completion. For example: 

(13) We've studied English.... 

A. for one year B, since last year 
*C. one year ago 

B. Choose the correct response. 

The students hear a statement or question followed by 
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three responses, only one of which is correct. A great vari- 
ety of structual and vocabulary problems can be tested in 
this way, the following examples being typical. 



(14) 


Are the words difficult? 
B. Yes, it is. 


*A. No, they aren't. 
C, Yes, they do. 


(15) 


Is the train fast? 

B. No, it's quick. 


A. No, it's early. 
*C. No, it's slow. 


(16) 


Did you see Mary? 

*B. Yes, X saw her. 


A. Yes, I saw it. 
C. Yes, I see her. 


(17) 


Did you buy the tickets? A. Yes, I bought it. 

*B. Yes, I bought them. C. Yes, I bought one. 


(18) 


I’m looking for a policeman. 

*A. There's one dver there. 

B. He's over there, C. They're over there. 


(19) 


Did you get any information? A. Yes, I got one. 

B. Yes, I got a few. *C. Yes, I got a little. 


(20) 


Are you going to buy a car? 

A. No, but I intend to. B. No, but I have to. 

*C. No, but I'd like to. 


(21) 


There's a fire in the kitchen, 

*A. Please put it out. B. Please put it off. 

C. Please leave it out:. 


(22) 


Is that your textbook? 
*B. Yes, it's mine. 


A. Yes, it's yours. 

C. No, it isn't yours. 


(23) 


May I sit here? 


A. No, not at all. 



B. That's very kind of you. *C. Please do. 

A variation of this technique is to have the students 
listen to the answer to a question and then choose the ques- 
tion corresponding to that answer. An example involving in- 
terrogative words is: 

(24) Mr. Tanaka goes to Tokyo three or four times a month. 
A. Where does Mr. Tanaka go in Tokyo? 

*B. How often does Mr. Tanaka go to Tokyo? 

C» When does Mr. Tanaka go to Tokyo? 

C. Choose the correct sentence based on a situation. 

The situation may vary in length from a single sentence 
to a short paragraph. The students listen to it one or more 
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times after which they hear three sentences only one of 
which fits the situation. The following examples illustrate 
this technique. 

(25) Not many of the books are expensive. Therefore; 

A. A few of the books are cheap. 

*B. Most of the books are cheap. 

C. One of the books is cheap. 

(26) Ken laughed at Bill's joke. 

A. Bill was amused. B, Ken was amusing. 

*C. Ken was amused. 

(27) Bill met Ken on the campus „ "Where are you going) 

Ken?", he asked. 'I'm going to class," replied Ken, 

A. Ken told Bill where to go. 

B. Ken told Bill to go to class. 

*C. Ken told Bill where he was going. 

(28) I studied in the library until lunch time. Then I 

went out. 

*A. I studied in the library before having lunch. 

B. I left the library after having lunch. 

C. I had lunch before leaving the library. 

(29) I wanted my brother to take up English, but he decided 

to study French, 

A. I got my brother to take up English. 

*B. I urged my brother to take up English. 

C. I persuaded my brother to take up French. 

(30) John is sleeping in class again. 

A. He must be studying. 

B. He couldn't be tired. 

*C. He should be paying attention. 

(31) John's car cost $2,000, Peter’s cost $3,000. 

*A. John's car isn't as expensive as Peter's. 

B. Peter's car is cheaper than John's. 

C. John's car cost more than Peter's. 

(32) It's not advisable to talk about politics here. 

*A. Politics should be avoided here. 

B. Politics cannot be neglected here. 

C. Politics may be discussed here. 

It is also possible to make one situation the basis for sev- 
eral questions. The following illustrates the testing of the 
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students knowledge of the possessive. 

"George Kennedy is a businessman. His wife's name 
is Ellen. His partner is Walter Moore and his- 
secretary's name is Miss Susan Green." 

(33) What is Walter Moore? 

*A. He's a businessman. B. He's a secretary. 

C. He's Mr. Kmnedy's partner. 

(34) Whose boss is Mr. Kennedy? 

A. Ellen's. B. Walter's. *C. Susan's. 

(35) Who is Ellen's husband's partner? 

A. Susan. B. George. *C. Walter. 

In a problem of this type it would be necessary to let the 
students hear the passage at least twice before having them 
choose the correct answers to the questions. 



THE CHOICE OF CONTRASTING ELEMENTS 

In preparing a test of this nature, it is important to 
keep the choices available to the student within a definite 
structural range and not to confront him with widely dis- 
parate forms, many of which could be eliminated by common 
sense alone. Suppose, for instance, that the example (30) 
above were presented as follows: 

John is sleeping in class again. 

A. He must be studying. 

B. He couldn't be tired. 

*C. He is a lazy student. 

In it's original form, this problem was designed to 
test the student's understanding of the pattern: MODAL + BE 

+ COMPLEMENT. However, by applying common sense, most stu- 
dents would know that, in the new form, C was the correct 
answer without having to know the required pattern. The same 
could be said of example (26) , if the third choice were 
given as, for example: The coke was funny. 

It is necessary, therefore, always to keep in mind the 
structures which the learner confuses because of interfer- 
ence from his native language, and it is precisely those 
structures which must be tested. When the text being used is 
already structured with a view to teaching Japanese stu- 
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dents, — as should always be the case if a genuine linguistic 
approach is employed — the teacher can easily make a list of 
the patterns already covered, and it only remains to put 
them in a suitable form for testing by using the techniques 
illustrated above. 



TECHNIQUES EMPLOYING VISUAL AIDS 

The techniques considered so far have been completely 
oral in that the student does not require any written mate- 
rial and merely reacts to what he hears by marking a letter 
on his answer sheet. Various modifications can be employed 
which introduce visual aids. 

The student can be given a picture and required to 
choose correct answers to questions related to it. For ex- 
ample, a picture illustrating an arrangement of objects 
could be used for testing command of prepositions of place 
and position. The student can be given a written passag* to 
read and then choose answers to questions given orally. 
Again, a series of problems can be given which involve one 
of only three or four standard answers. For example, in 
testing the ability to form simple questions from state- 
ments, the student can be given an explanation sheet con- 
taining the information: 

A = Is • • • ! B = Are . . . ? C — Does . . . ? D = Do . . . ? 

The student would then hear a series of statements which he 
would have to convert mentally into interrogative form. If 
the question he forms begins with IS, he would mark A; if it 
began with ARE he would mark B, etc. 



TESTS WITH ONLY TWO CHOICES 

Certain structures are not so amenable to being put in 
the form of problems with three alternatives. A typical ex- 
ample is the pattern involving a giver and a receiver with 
the prepositions TO and FOR. For example: 

(36) Mrs. Conroy made a cup of coffee.... 

A. to her husband *B. for her husband 

(37) She offered a cup of coffee.... 

*A. to him B. for him 
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When this type of problem is included, allowance must of 
course be made for the higher statistical probability of 
guessing correctly. 



WRITTEN ANSWERS TO PROBLEMS GIVEN ORALLY 

This brings the technique of dictation one step further 
in that it involves structural manipulation in addition to 
a clear understanding of the cue sentence and the ability to 
write it. There is virtually no limit to what can be tested 
by this technique! active to passive transformations being 
only one of hundreds of possibilities. It is hardly neces— 
sary to point out that it should be given only to students 
who have made some progress in writing and that the problems 
should be selected so that what the student has to write is 
not overly long. 



PREPARING THE TEST SCRIPT 

The teacher first lists in order of importance the sep- 
arate grammatical structures and related vocabulary items to 
be tested. The number of questions to be set must then be 
determined and the test planned so that more questions are 
allotted to important structures and fewer questions to 
structures of relatively minor importance. The time allowed 
for the test must also be taken into account! It is safe to 
allow about 30-seconds for each question. Each section of 
the test must be preceded by clear and consise explanations 
illustrated by examples, so that che students will know ex- 
actly what is expected of them. Fcr students at the begin- 
ning level, these explanations should be in Japanese. 

Each of the questions set should conform to the follow- 
ing criteria: 

(1) It is designed to test the students understanding and 
recognition of a definite structure and accompanying 
vocabulary problem already covered in classwork. 

(2) There is no ambiguity in the alternatives given., It may 
happen that under certain circumstances two of the al- 
ternatives could equally be chosen. This must be avoid- 
ed. 

(3) The question does not test artificial correctness. For 
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example, the student should not be asked to choose be- 
tween I wish I were and I wish I was , forms equally ac- 
ceptable to educated speakers. 

(4) The question is not given in such a sophisticated form 
that it requires ingenuity not related to language 
ability. 



RECORDING THE TEST 

It is assumed that the recording equipment and facili- 
ties available are sufficient to produce a tape which can be 
heard at least as clearly as the teacher's voice in the 
classroom. Most schools in Japan now have machines and con- 
ditions for meeting this requirement without difficulty. 

If possible two voices should be used; one for giving 
the question, the other for giving the various choices. 

A normal conversational speed of speech should be main- 
tained, as any attempt to emphasize the point being tested 
by artificially distorting the natural rhythm of the sen- 
tences would defeat the purpose of the test. 

The correct timing of pauses on the tape is important. 
Sufficient time must be given for the student to react to 
the question, but if the time interval between the choices 
is too long, the student may have forgotten the first by the 
time he hears the third. In most cases the student should be 
able to mentally produce his answer first and then check it 
against the possible answers as they appear. Therefore the 
main pause after the question should give the student time 
to mentally formulate an answer after which the alternatives 
should be given fairly rapidly at intervals of one or two 
seconds . 



EVALUATION OF THE TEST 

Once the test has been prepared and recorded, it still 
remains to check its reliability. In other words, it should 
give reasonably consistent results when taken by several 
groups of students who have reached the level of proficiency 
for which it has been designed; and these results should 
conform statistically to the bell-shaped distribution curve 
obtained when the number of students is plotted against 
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their performance. If it is found that a certain question is 
consistently missed by almost all students, that question 
should be replaced. It is recommended that the teacher who 
has prepared the test try it out on a group of his col- 
leagues before giving it to the students. The results of 
this experiment may sometimes be surprising and give a good 
indication as to whether or not further modifications are 
needed. 



WHAT ABOUT SPEAKING ABILITY? 

To what extent can aural comprehension tests as des- 
cribed above be used as assessments of a student's ability 
to speak English? Is there a direct correlation between his 
ability to understand to speak? In order to answer these 
questions accurately, more data would be required. It would 
be necessary to construct speaking tests of the same degree 
of reliability as that of a number of aural comprehension 
tests corresponding to the same levels of proficiency. The 
results obtained would then have to be compared for a suf- 
ficiently large number of students so as to give statisti- 
cally reliable data. The difficulty is, however, that in 
the present state of the art of testing, there are no highly 
reliable speaking tests which can be placed on a par with 
corresponding aural comprehension tests. 

Nevertheless, there are speculative and subjective ex- 
periential factors which do point to a close relationship 
between the grades obtained in a good aural comprehension 
test and the ability to speak. The skills of understanding 
and speaking are closely related and are taught together. 
Understanding usually develops faster, but, given satisfac- 
tory teaching, it is reasonable to assume that the speaking 
skills also develop at a related pace. Furthermore, the 
teacher in the classroom should be able to make a reliable, 
though not quantitative, judgement as to his students speak- 
ing ability and to classify them to some extent in the order 
of this ability. If aural comprehension tests of grammatical 
structure give results in agreement with such classifica- 
tions, this is good evidence for their ‘validity as assess- 
ments of skill in speaking. Over a period of five years the 
writer has had the opportunity of subjectively assessing the 
performance of small groups of students in the classroom and 
comparing this assessment with the grades they obtained in 



15 



The Modem English Journal 



tests of the type described. Such comparisons have clearly 
demonstrated that there is a close correlation between a 
student's ability of speak and his performance in these 
tests. 
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Pattern Brill'. From the Peace Corps Volunteer by way of 
the Texas FLA Bulletin we report the following real life 
pattern drill, quoting a Peace Corps volunteer teaching 
English in Thailand: "Teacher: This is a chair. Students: 

This is a chair. Teacher: This is a mango: Students: This 

is. a mango. Teacher: Table. Students: This is a table. 

Teacher: That. Student: This is a that. Teacher: No, think 

please. Student A: This is a think please. Teacher: No, 

a thousand times no (pause). Very Bright Student: That is 

a table. Teacher: AhS Correct Eye. Student B: I is 

actable. Student C: I am a table. Exit teacher." (From 

Hispania Dec ember 1963.) 
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